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mceeutlv^  Tunmary 

There  has  been  an  increasing;  emphasis  placed  on  risk 
analysis  In  DOT.  DCDD  5000.1,  entitled  Acquisition  of 
"ajor  Defense  'ystems,  has  directed  that  ..eanir.rf  ul  rela- 
tionships between  need,  urrency,  risks  and  wortn  be  es- 
tablished. As  early  as  the  conceptual  phase  of  a progra  , 
the  individual  Service  is  to  conduct  a risk  assessment  be- 
fore appearing  at  D3ARC.  DCD  representatives  at  D5ARC  are 
looking  fer  evidence  that  program  risks  have  been  identi- 
fied and  that  appropriate  action  is  olanned  for  their  reso- 
lution. "‘o  obtain  a risk  assessment,  the  individual  Service 
roes  threurh  a process  of  estimatin'-  or  subjectively  determ- 
inin'- the  probability  that  a specific  interplay  of  perform- 
ance, schedule,  and  cost  will  not  be  obtained  along  the 
planned  course  of  action.  Although  these  types  of  qualified 
"engineering  judgment"  analyses  are  creditable,  the  kind  of 
risk  analysis  presented  in  this  case  study  will  form  the 
basis  for  a quantified  type  of  analysis. 

The  case  study  is  bared  on  an  actual  risk/declsicn 
analysis  performed  by  the  Army's  LATCE  Project  Office  at 
Redstone  Arsenal,  Alabama.  The  methodology  developed  for 
that  study  was  successful  ir.  identifying  a program  option 
for  each  production  buy  that  would  accomplish  the  program 
objectives.  The  raetv  lory  embodies  the  concept  that  the 
analyst  should  be  able  to  use  "fact"  documents  (present 
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statu  rts;  emonstral  statu  reports;  consultanl 

repor  ; results  of  a s nti  bj  lnd  pendenl  authors; 

etc. ) to  conduct  an  h iris  tic  risk  malysi  and  1 clsl 
on  hli  r<  rra n.  Th  heuristic  analysis  juantl- 
fies  th  probabilit;  f 11  thresholds  1 :h<  dul  , 

cos t,  and  i r ft  naano  associal  1 1th  i<  iisit  . Th  1 - 
cl  si  or.  analysis  treats  the  sensitivity  of  decisions  and 
trade-offs  and  st  ites  th  optimum  decision  or  approach  to 
• ■ - . l ..  t he  decision  ans  sis  is  to 

reduce  the  risks  associated  with  each  option  to  a co:v.ion 
base,  such  an  dollars.  II  should  be  stressed  thal  th<  de- 
1 l . aker  ••  • still  .*  1 • . tally  assess  the  Mun  juai  1 1 - 
■ in  ' i.  ca  s<  i thi  fina]  eclslon. 
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The  J fiber  ' Is;; lie  Case  :t  :dy 

As  Col.  Gil  Griff  left  the  conference  roo:n,  the  words 
of  ler.pral  .n.iff  (F*  for  the  _aber  Missile  iystea)  were 
still  rinring  In  his  ears.  "This  pro ’rao  has  reached,  a 
point  in  Its  life  cycle  where  It  should  be  ready  for  pro- 
duction. With  DCA'C  III  only  two  months  away  (dept.  1,  1973) 
there  are  still  s one  requirements  that  have  not  been  met 
on  this  system.  "here  has  to  be  some  way  of  identifying 
the  risks  involved  before  entering  production.  fe  must  be 
able  to  recommend  some  valid  approach  to  the  production 
phase  at  D3A3C. " As  deputy  prorram  iana~er,  Col.  Gruff 
nad  been  directed  to  consider  the  use  of  risk  analysis  as 
a management  tool  in  arriving  at  a viable  option  for  the 
production  phase. 

5ac>.  -re uni 

Saber  is  a hirhly  mobile,  surface-to-surface,  ballistic 
missile  sy  te-.  which  will  provide  a nuclear  and  non-nuclear 
capability  in  (reneral  support  of  Army  forces.  The  Army 
plans  for  the  3aber  to  reolace  the  Ilavipator  "issile  Gyst.a, 
which  has  been  deployed  for  ten  years.  The  target  for 
overseas  deployment  (C3D)  of  3aber  is  Sept.  1,  1°74.  The 
initial  puidance  system  has  been  specially  deslpned  to 


This  paper  represents  the  views,  conclusions  and  recommend- 
ations of  tne  writer  and  does  not  necessarily  reflect  the 
official  opinion  of  the  Defense  Systems  Management  School 
nor  the  Depart uent  of  Defense. 
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~i v*  Saber  ' hirh  de~ree  of  accuracy. 

'.aber  has  been  In  development  for  four  years  and  has 
completed  only  the  er.rinetring  tests.  Performance  was 
found  to  bt  less  than  required  in  two  areas: 


Achl eved  iequi  red 

In-flight  reliability  (prob)  .7(14/20  shots) 

Accuracy  (’.11s)  1.5  1.0 


ether  systi  a charao Leri -tics  are: 

.La are  (Kilometers)  5C-1000 


Wei  ht  (missile  + vehicle; 
lbs) 


10,000 


Launch  vehicle  Cross  country  mobile,  armored, 

amphibious  vehicle 

The  nuT.be r of  systems  required  to  counter  the  projected 
threat  are  5,^00  Saber  missiles  with  warheads,  and  100  launch 
vehicles.  The  laber  requires  only  5,000  troops  for  su^nort 
in  the  field  whereas  the  Navigator  System  requires  1" , 000 
troops. 

Present  Situation 

Col.  Gruff  had  access  to  the  development  documents 
for  the  Navigator  System,  so  he  began  to  analyze  some  of 
the  data  Involved  as  well  as  the  decisions  made  on  that 
program.  There  was  no  indication  that  a risk  analysis  had 
been  perforned  prior  to  entering  the  Production  Phase.  3ut 
the  system  had  been  in  the  field  for  a long  time  and  the 
pervalling  oolltical  climate  at  the  time  it  was  developed 
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n ; lowed  the  incurrence  of  considerable  exren.  e to  Insure 

th  • lance  was  optl  ized.  Ool.  Gruff  knew  that  tl  r 

hsd  charged.  The  dollar?  were  United  and  no  rrogram  would 
nans  D1A1C  without  the  service  havin'  raid  considerable 
attention  tc  the  risks  involved  - risks  in  cost,  sched  ,le, 
anci  rer4  cr:rance . T susliy  sr*  analysis  oz  iht?  risks  i r.  a src- 
raa  wo  ;ld  aanifest  itself  in  the  identificaticn  of  several 
options  which  management  could  consider  before  proceedin.  . 

About  that  time,  Col.  iruff  received  a call  fro-.  Col. 
avi  ■.  whc  . . its  Prc  ri  L(  . nt  "oni tor  in  tht  entagon. 

Col.  :ava,-e  related  several  of  his  impressions  of  C3L  aan- 
nent,  since  he  had  sat  in  on  several  of  the  DSARG  sessions 
recently.  Apparently  there  was  wide  usare  being  made  of 
risk  analysis  in  de terT.ir.ing  whether  a program  was  ready  to 
proceed  into  the  next  phase.  'avare  indicated  that  there 
were  a couple  of  program  'anacers  who  had  defended  their 
programs  q i te  successfully  in  this  area:  Col.  Harvey  Coal, 
ar.d  Col.  Curly  Huff. 

Later  on  that  day  Col.  Gruff  decided  to  call  the  pro- 
•'ran  managers  that  Sava  e had  mentioned  and  Invite  t over 
for  a chit-chat  on  risk  analysis.  Col.  Coal,  the  program 
mar.arer  for  the  nafu  Missile  System,  Just  happened  to  be 
available  that  very  afternoon.  His  program  office  was  at 
Redstone  Arsenal,  but  he  was  in  town  as  a r-uest  lecturer  at 
American  University  on  decision  analysis.  In  the  conversa- 
tions that  followed  in  Col.  Gruff’s  office,  Col.  Coal 
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ex;  lalned  how  op  had  used  r!  sk/decl  elon  analysis  in  hi? 
ovrr  vrorra  . -naf  * was  a surface-to-surface  fissile  syste  . 
t ploy*  n.  fl  .id  ildar.ce  with  a '.id-course  prorra-  ed  flight 
profll«  and  It  er  terra! nal  he  j1 n ( na  icii  i rang*  50<  lies)  . 
Col.  Coal  i. ad  acquired  the  pro-vra-  approximately  r.ir.e  icr.ths 
before  ; was  due  to  enter  production.  Based  on  engineer- 
in,  tests  which  had  Just  been  completed,  he  i mined  lately 
initiated  ? risk  anal  y el  :•  stud;.  to  determine  what  action  he 
: hould  take  concerning  the  production  decision.  Figure  1 
show  thi  cl  si  : tret  that  Col . Coal  developed  In  order 
to  raphically  depict  the  three  ortiens  considered: 

Crtion  I 

Start  production  rt -ht  away. 

Cptior.  II 

Valt  or.e  year  to  start  production  ar.d  continue 

H v D for  si>  more  months. 

Cptlcr,  III 

Valt  two  .ears  before  startin  production. 

Tnere  were  two  subeptions  available  for  each  of  the 
^ options: 

r iuboption  A.  Accept  the  performance  of  the  produced 

item  if  performance  was  within  ’'inronatle  bounds. 

I -.easonable  bounds  in  this  rropram  were  considered  to 

be  as  low  as  65  per  cent  reliability  and  1 . c mils  CEB; 
however,  if  the  performance  was  within  these  limits 
_ but  below  the  specified  values  (.75  reliability  and 
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. ils  CEP) , 10 r ' ' 1 rcha 

•over  5.  red  ber  tar  ts. 

3uboptloJ  . letrofit  all  th<  oissll  » at 
' . • aft  r r<  luction  was  start  1. 

Col.  •’  f . •!  ' ' : yc  . con  1 r olfi  lnl 

ctj  ri  ht  awaj  wh  a ; cnew  yo  i wer<  havli  probl<  ns 
■ ■ f th  perf orraanc  ara  el 

. ; plai  ■ ' ha  1 tl  e a Iva  ntages  of  * • llati 

n . ctlon  o-ahead  Included  the  following: 

1.  Snafu  would  rc-rlac  existin  field  . systems 
requirl  soldiers  per  weapon.  ifhen  f 

operational,  3nafu  would  require  9»00(  troops; 

tin  systems  (Shamrock  and  Conqueror)  r - 
ired  tr  s.  At  510,000  per  year  per 

, Snafu  W(  Id  r:av  ■ sill  is..  ly . 

. It  would  delete  the  requirement  for  onths  of 
addi  ti  a]  R D by  the  eontr  ''nr. 

skilled  ] SOCiated  with  the  7.  & D 

'ffort  could  bo  transferred  directly  into  the 
uanufact  rinr  phase  and  thus  the  r -o  rra.i  would 
enjoy  the  advantage  of  personnel  continuity. 

. Joal  then  explained  that  the  risk  analysis  re- 
sulted In  a listinr  of  the  cos L , schedule , and  perfor  ance 
characteristics  considered  and  the  probability  that  the 
expected  values  (the  jean  of  the  distribution)  of  the  char- 
acteriuticn  would  not  be  net.  (Table  1) 
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Annlysi 


Tor  Snafu 


r— 

Cuarac  l - r'. .: ' T.  cs 

Cpt 

ions  .a  n n ?.  ::  1 

T,  ^ 

I 

I-A  | 

’ _ 

' 1 1 -A 

: i - 

- 1 . “ 1 

i * «.  — 

1 ro  ra cos"  : (In 
r.lllior.3  of  dollars) 

1 

1520 

1625 

I 1566 

164.. 

‘615 

0 4 • 

: ■ l eplo ; 

(in  ;on  ths) 

21 

24 

7 X 

J > 

45 

46 

(probability  of  not 
deploy! n - wi th5  . 

21  months 

l 

. 

. 

i 1 0 

l.'y 

1.2: 

1 

1 . 0! 

Probability  f 
c edir.-'  cost  ~cal 
c p ;55°  million 

• r 2 

l.oo 

; .00 

, 

1.0 

1.0! 

1.00 

1 

System  reliability 

1 

. 6Q 

?n 

. 7 5 

.75 

.7 

7 

— 
SI 1 

.50 

.50 

1.00 

1. 

System  accuracy 
(in  mils  CnP) 

1.5 

L. 

1 • 2 

1 N 

L • 

1.0 

or 

Probability  of  :.oet- 
i.a  • or  exce  ;d  1 n * 
accuracy  of  1 rail 

0 vr 

.10 

1 

. 20 

.50 

.60 
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hat  look*  ' i r lve,  Harvey,  but  how  dt  you 

calculate  the  1 ro  ha  bi  1 1 * les?  ” 

MI  t depi  ■ . pari  replied  Col.  Coal . 

' For  lnstar.ce,  the  probability  of  not  deploying;  within  21 
a:  or.t'r.  ■'  ftarrt*  of  CCD)  is  rather  civil  us  for  Cutler.  II  ar.d 
III , he  pro ba bill  tier  for  I -A  and  I - • ri  julred  a careful 

. alysla  >f  tl  ncertalntlea  r ro  mdli  t the  • events . 

“hey  were  deter  inert  'ey  carefully  con- * r ictln  a distribu- 
tion ror  each  event  (Figure  2),  the  probabilities  fcelr . ' 
rt  rived  as  shown.  The  other  characteristics  were  handled 
ir  a like  manner. " 

As  the  discussion  progressed,  Col.  Coal  shared  with 
hie  sc  e of  the  rrc  .nd  rules  one  miyht  use  ir.  constructln 
the  distributions  of  the  cost  and  time  characteristics. 

Cost  ■ I round  Rule-; 

1.  All  costs  Include  prior  year  costs. 

2.  Costs  are  discounted  at  the  rate  of  10  ^ Der 
year  tc  equate  all  costs  to  present  year 
purchasing  power. 

1 . Future  costs  are  based  on  projected  remain- 
ing costs  for  procurement,  R & D,  and  retrofit 
costs . 

The  following  activities  will  generate  costs: 

1.  Research  ar.d  Development  costs  (R&D)  to  in- 
crease system  performance  to  meet  require- 
ments . 
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. isslle  (lncl 
co  sts . 

7 . In  inch  vehicle  costs. 

A.  Retrofit  cost?.  It  is  assumed  that  retro- 
fit >'5  11  increase  system  rerformar.ee  to 
required  values,  but  at  the  ri sV  of  in- 
creas'd cost  and  time. 

5 .e  Ire  nc  .ili-s  : 

1.  Tomir.al  task  completion  times  era  reed  and 
are  add! ti ve. 

2.  The  expected  time  to  start  retrofit  is  one 
year  from  the  decision  date.  The  cause  of 
lew  : erforrsance  will  be  known,  and  the  i - 
moved  performance  is  exoected  ic  be  demon- 
strated by  that  time. 

1 . etrofit  ti"p  includes  redesim  time,  test 
time,  rer.anufacture  time  and  retrofit  time. 

I terns  1 through  7 immediately  above  cause  schedule  slippage 
and  are  r.  fei  red  to  as  time  generators.  The  time  -er.eratcrs 
are  arrived  at  using  known  °acts.  If  facts  are  not  available, 
best  estimates  from  experts  lr.  engineerin',  programming  and 
scheduling  must  be  used.  The  estimates  are  to  be  as  re- 
alistic as  possible  (tolerances  should  be  Indicated  as  well 
as  probabilities  of  occurrence),  and  the  various  options 
treated  consistently,  Various  techniques  are  available 
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for  soliciting  and  evaluating  expert  opinion. 

Col.  Gruff  realized  that  he  would  have  to  be  very 
cai  ef ul  in  the  assignment  of  his  peorile  in  order  to  do  a 
el  liar  stidy  on  3nef'i.  .ie  would  need  reorle  Icnowled rable 
ir.  the  areas  of  system  assess  lent,  cost  accounting  and 
es tlmatim*,  and  scheduling-. 

"Cid  you  -Lake  yo  decision  fiver,  only  the  risk  analy- 
sis data?"  asked  Col.  Gruff. 

"Go,  because  those  values  do  not  provide  any  direct 
assistance  in  chooeir.r  the  most  attractive  option,"  replied 
| Col.  Coal.  "A  decision  analysis  was  then  performed  ir.  order 

to  transpose  the  expected  values  for  all  the  parameters  to 
a common  base.  The  base  that  is  best  understood  is  the 
dollar  base,  and  that's  the  one  we  used." 

Col.  Gruff  found  that  one  approach  was  based  on  a single 
computation  of  dollar  costs  isinr  the  expected  valjes  in 
Figure  1 and  trade-off  criteria  (or  fibres  of  merit)  in 
order  to  equate  risk  to  cost.  Criteria  such  as  the  follcw- 
used  for  inafu:  (Table  ? shows  overall  expected 

The  time  cost  Is  the  difference  between  main- 
taining the  Ghamrock/Conqueror  Syste.is  in  the 
field,  ar.d  maintaining  the  Gnafu  - multiplied 
by  the  months  difference  in  deploying  Gnafu  for 
the  different  ootions  (PI  months  is  the  schedule 
goal ) . 


ing  were 

( 

costs ) 

I l- 

f 
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Table  2 


Option  Costs 


r%n  Of'  T ^ T 1 

L»  L o * * 1 x 

i 

I-A 

I-B 

II-A 



11-3 

III-A 

T T T 1 

ill  — —*  | 

- 1 

Time 

0 

20 

5.3 

40 

I 

41.7 

1 ro  -ram 

0 

105 

46 

126 

93 

126 

lei  i abi  11  ty 

17.4 

0 

0 

-7.4 

-10.4 

Accuracy 

133 

0 

47*^ 

. 

0 

0 

-9 . - 

mm 

1 19*0 

H 

H 

| Total  Expected 

1 Option  Cost 

— 

670. 4 

630 

f 4?  . • 

. 
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I 


I 

I 

I 


( 

I 

I 

I 

f 
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. Thi  ro  pan  co  1 l on  ;h<  remisi 

that  eacl  1<  Liar  fpent,  above  the  least  costly 
O] tion,  would  be  a dollar  cost, 
leli  lllty  cost  Is  c >m  under  the 

assume  lien  that  one  Is  ir.diffc-rent  between 
system  reliability  and  the  number  cf  mi.  siles 
purchased  a:-  Ion  as  the  reliability  doe.: 

I Low  the  reas  nabl<  nd . - signed. 
Reliability  cost  = yisslles  required  X 'fissile  ICC 
•>here : 

lissili  ! r<  i .75  X 1 - . 

Assessed — tfeHTTty  " 

!1  sll  LCC  - ‘ . , (del  rn lned  by  cost 

experts) 

To  the  decision  maker, this  calculation  equates 
risk  to  cost  and  does  not  imply  an  actual  in- 
ti nt  to  purchase  the  additional  missiles. 

Acc  racy  costs  are  approached  much  the  same  way 
as.  reliability  costs.  A 1.5  mil  accuracy  is 
ised  s an  example.  Assuming  circular  norm- 
al distribution,  the  object  is  to  find  the  prob- 
ability of  a 1.5  mil  accurate  syste  i J apactin 
missiles  within  a 2 ail  target  radius.  ( 1.1774 
sigma  = radius  of  a . E probability  circle) 

1.1774  sigma  = X 

1.5  mils  CEP  l.C  mils'  i'hl 

X - .7849  Sigma  .265  probability 
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'."he  aber  of  missiles  required  to  insure  that  50  t will 
. ' ' thi  - . ill  r iiu:  1 

•,  ..  _ X 

« - ' • 

X - l4l:  missiles 

: 'il  a v suae r,  an  indifference  to  accuracy  and 

• , ro cured  without  lmplyin  an  act  la 

this  any  ore  il  11 

"The  option  costs  fall  out  neatly  usl  1 r ch," 

: . ■ ’ l . "There  Is  ] ro  1 b 1 y i more  rJ  oroui 

rc  ic:  .available,  i n't  there?" 

1 ' . tree  j roach,  Sil.  Instead  of  isln 
V;nt  the  e;:-  acted  value  to  compute  the  final  cost  of  each 
c; tion,  tne  actual  distribution  for  each  parameter  would  be 
• : lx  the  tree.  Sufficient  .,oa  si  tivi  ty  van  ;•  v-  . 

by  selection  of  only  five  points  from  the  distribution  and 
these  points  are  then  used  to  create  the  event  fan  for  each 

■ In  the  tree.  The  same  distributions  used  to  create 
th<  ri  ml;  >is  matrix  ar<  ised.  5t  rtin  th<  t<  ni  1 1 - 
al  point  of  the  tree  network,  each  event  fan  could  be  aver- 
ared  out  and  folded  back  to  eventually  create  one  single 
tar  valut  for  that  particular  option.  This  valu 
should  be  approximately  the  same  as  that  assessed  usinr  only 
the  mean  ( expected)  value  for  each  para  . r. " 

Pi  ure  hows  the  tree  network  for  Option  I-A.  The 
chance  fork  for  ,?(50r)-ltr  under  program  cost  indicates 
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ther  1 . ••  llity  that  pr  pi  wil 

This  ] ■ l Le  ' to  a cost  savings  of  $3  K. 

A solutioi  tion  I- A resulted  in  the  following  costs 

li valent  tc  1 lnv  v : 

Crtion  I - A ( : ::illlcns:) 


i'.nnimw: : 


ixsecte  i 


• 


. 

. ■ 


, Jol.  Jruff  called  1 1 

an:  parceled  ou  t the  work  as  si  -nmentc  in  order  to  construct 
■ ri  ieci  1 ysis  on  Saber.  He  r«  lnd  i his  peopl 

that  th<  cost  -oa3  for  the  program  was  still  5500  K. 

Ie  also  briefed  his  staff  on  the  time  and  cost  assump- 
ti nder  rhich  the;  wer  t work  in  developing  the  nec- 

essary data  (same  assumptions  as  those  used  for  the  Snafu 
ii  sile  syst  ).  able  3 show  the  data  submitted  by  his 
; tl  i tii  ec  pie.  71  ;ur«  s and  ' w re  lev  Lo] 

;he  same  group  in  order  to  simplify  the  relatio  l hi]  be- 
tween the  levels  of  accuracy  or  reliability  predicted  and 
the  number  of  additional  aissiles  which  to  he  ri  :k. 

FI  -ure  6 shows  the  decision  diagram  that  Col.  Sruff  and 
his  deputy  developed.  The  diagram  shows  three  options 
which  (when  analyzed)  will  aid  General  Snuff  in  deciding 
how  he  will  present  his  program  at  DSA1C. 

Col.  Cruff  had  explained  to  his  system  assessment 
people  the  value  of  a decision  tree  approach.  LCOX  Con  Herti  , 


Table  3 
Cent  Data 


I 


Cost  I-er  Tactical  Assile 

Cost  Category 

Discounted  LCC 

Missile  Procurement 

: 62,750 

Missile  Depot  Maint.  & Trans 

islle  Spares,  Provisioning 

11,140 

Class  V 

13,250 

Missile  Storage  and  Support 

? A 360 

^rtr;5^ 

Ire.  ran  Costs 

Previous  Fro  -ran  Costs 

O 

O 

■SF 

Procurement  costs  (ir.clJdes 
vehicle) 

eno  m 

R & D Frorram  Cost 

;50  M/Yr 

Retrofit  Cost 

34,000/Missile 

Cost  Per  Man  Year 

TIC , 000 

I 

I 

I 


* 


I 


' ‘ - 1 i > 3afcer 

. 

*e:j  : 1 re-.ents 


t - i ".1  rel 1 abl 1 

■ . \ r. e produc  tlon 


1 K 1 = 

Awar  : 

■ tart  roductlor 

1 

?1 r s iab« : 

lellv  r*  1 to  At 

deployin  g / 

■ ta*i«  and  nnaalc 

. . 1 rli  1 vision  Head,  latched  on  to  the 
■ way.  Ft  hi  round,  wt  abl<  t<  recal 

owledge  about  sampling  • lstrll  itlons. 

1 res  7,  , n • were  submitted  to  Col.  @ru ff, 

( lc<  tad  ill  the  lata.  Col.  Gru ff  ere  ited  ecial 
r<  i]  ii  order  to  facllitat<  the  analysis.  LCOL 
Her • • v."  named  chairman  of  that  *rour. 
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Table  h 


ion 


I 


7 1 "j  re 

Distribution  of  th<  Fro  r-.*.  Cos*. 


. 


C • tior: 

I 

4n0 

500 

510 

520 

53C 

0 p t i o n 

II 

5^0 

550 

560 

570 

560 

Option 

III 

550 

560 

570 

560 

590 

Pro,-ra::i  Coat  ($  r".lllions) 


Areas  for  Consideraf ior 


* W ' Sniff  pursuin  the  righi  course  of 
nc  lion? 

* Does  Col.  Gruff  have  enough  Information  to 
conduct  a ri sk/deci sion  analysis? 

* How  realistic- are  the  criteria  tbal  >•  r 
to  quantify  risk? 

* What  are  the  advantares  to  conductin'  a risk/ 
decision  analysis?  Disadvantages? 
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I 

Instructor  u'oter. 

1.  disk  analysis  If  a ■.  ar.a  ,-enen  t tool,  useful  In  present- 
lnr  the  alternatives  to  decision  makers  lr.  such  a way  that 
tne  tost  attractive  alternative  is  apparent.  Risk  analysis 
nay  be  thought  of  as  identifying  tne  risks  of  exceeding 
e.-tabllshed  thresholds  of  cost,  sched  .le,  and  technical 
pf  r fo rman  ce  . 

?,  A risk  analysis  is  often  thought  of  in  quail  tative 
terns,  but  it  car.  be  quantitative.  However,  the  worth  of 
*he  risk  analysis  is  Judred  primarily  by  how  well  the  reader 
car.  aval  -ate  it.  "'his  means  that  the  figures  used  :a;st  be 
do  ■ .er  ud  rder  for  the  reader  to  place  any  measure  o'* 

nfl  do  In  the  analysis.  Documentation  co.li  b<  satis- 
. >;•  ‘.1  developed  derivation  of  a quantity, 
vet  it.  i rl  k analysis,  some  assumptions  have  to  be 

' trt  elements  of  cost,  time,  end  performance.  Cne 
>f  tn-  , os  * 1 oortant  assumptions  addresses  the  interrela- 

: ' f ti.t  :>  elements  in  a risk  analysis;  1 . e. , increas- 
• ert  r~aar.ee  can  be  obtained  river,  enough  time  and 

. r 4 ! ‘ * r.  t r j ml  g . 

• . • -r.-'rt  o ’ s 1 and  2 are  the  solutions  to  the  risk 

* d ‘r.*  decision  analysis,  respectively,  for  the 
' 4 • ;-Kte  . ''’he  probabilities  of  achieving  the 

■»  of  the  parameters  for  each  of  the  ontlon; 

p y -t  ' • idered  '’omewhat  subjective  based  on  how  the 


I 


» * • 


tud<  it  J t rpr  t thi  tails  of  the  dlstributioas.  Atti  ih- 
o.  2 shown  the  solution  . -ir.  • th  et  val 

md  the  lutJ ‘ ’ ig  the  decision  tree  approach, 

"iven  the  additional  inf  or  mat  lor.  (the  xIti-  and  air.'  v-  " 

••  -j  ] fro n the  decision  trei  a pr  . , th<  ptlon  select- 
d r rra  personnel  could  bo  different.  For  instanc  , 
Option  II  could  well  be  selected  over  Cation  III  because 
therf  Is  an  opportunity  to  brin-1,  the  -rosram  in  at  a lower 
cost . 

5.  Schedule  risk  Involves  the  identification  and  quantifi- 
cation of  significant  sched jle  contributors  which  could 
• tend  v.  pro  ra  : resu  1 1 la  elayin  th«  . date. 

The  followin’  items  contribute  to  schedule  rl  : : 

"anufacturing  time  for  all  hard  rar<  it<  is. 

b.  Cxrected  tiae  and  need  for  retrofit  of  those 
'.tens  determined  to  be  major  contributors  to 
technical  ”isk. 

M'hese  times  include: 

(1)  3equali  ficatlon  time  for  desi  r.  chan.-es. 

(°)  Manufacturing  time  for  retrofit  parts. 

(3)  Installation  time  for  retrofit  parts. 

(4)  Time  for  verification  tests  for  desirn 
changes. 

c.  Overseas  surface  shipment  tine  for  hardware. 

d.  Overseas  unit  trainin'  time,  where  applicable. 
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fii  -hi  re1.  lability  and  acc  . racy  ire  considered  as 


I 


I 

I 

I 

I 

I 

I 

i 

I 


• clini  c a]  '1  ska  ■ J Lfyin  - 

thal  If  th  flight  li  . . r<  1 1 a ’ , it  rt.1]  b<  ve  ry 
jurat  . rest  rocedur  can  be  cons  t ructed  5 ch  t ha 
a reliability  circle,  assuraln  a r jcu  *- 

cy , Wo  ild  includi  . 99%  of  all  "relia  !'  lssl  s.  An; 
lissll  Impacting  outside  this  circle  is  consider  1 Fail- 
ure for  reliability  t<  stin  , and  is  not  part  of  the  accuracy 
populati 


The  decision  tree  for  each  option  of  this  case  study  is 
relj  tlv<  ly  • a >y  to  construct.  For  aorf  c<  ilex  trees  ther 
Is  a computer  program  called  ADTR3E  which  is  commercially 
availabl  t<  • ‘form  the  calculi  tions  on  t d cist  tr<  . 
.i'ith  the  aid  of  such  a program  the  solution  to  a decision 
tree  can  be  displayed  in  terms  of  emulative  probability 
versus  c ted  cost,  pn  bill t;  lensi  versu  - ecte 

cost,  or  discrete  probabilities  versu  s cted  cost, 
display  should  of  course  be  presented  in  the  manner  that 
will  be  best  understood  by  the  decision  maker.  For  this 
study,  only  the  maximum,  minimum,  and  expected  values  were 
computed  for  each  option. 

The  expected  time  and  cost  of  retrofit  can  be  determ- 
ined by  processing  the  data  obtained  from  data  sheets  such 
as  those  included  as  Attachment  I,’o.  >. 

0.  The  student  should  discover  that  as  the  risk/decision 
analysis  Is  constructed  In  this  case  study,  the  risk  that 
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1 

>• 


A 


J 


the  total  option  cost  will  exceed  the  basic  program  cent 


I 

I 

\ 

I 

I 

* 

I 

I 


( 10  I',  for  the  lowest  cost  pro  -ran)  has  been  computed  on 
: . . risk  basis.  [f  th  pected  iral  ■ - • lecisi 

ad  n<  r LI  1 In  on  ption  clearl;  be- 

in  re  ■ si rable,  idditi<  lal  tr<  nt  rfo  . . h 

b<  . ary.  Other  treatments  ia1  could  be  5 1 at 

■ hi  r point  ire  we  i -htlnr  and  utilities. 

Tei.-hti'i-  would  involve  decidin'-  if  cost  should  have 
, L ss,  or  ;re  ter  reight  fch  sch  idu]  or  perf  ■ ce. 
The  utility  approach  would  consider  ho*-’  much  each  of  the 
ras  w rth  the  ecision  naker,  A r latl v - 
standard  could  be  established  by  considering  the  parameters 
airs,  i.e.,  cos  t '•  * le,  time/ performance,  and  cost/per- 
f 0 raanc e s p ai r s . 

A disadvant  • in  sin  a welghtin  factor  is  .■  in- 
dividuals wei  -ht  the  parameters  differently  and  this  lends 
Itself  to  dispute. 

. . rhe  risk  costs  used  for  reliability  and  • cc  ra 
based  on  purchasin'  the  additional  missiles  necessary  to 
shiev  th  iff  ctlveness  that  a system  -nee  tin.-  the  perform- 
r : ■ ' ■ ■ . - ild  provide.  Th<  decision  maker  ;<  il 

elect  to  purchase  the  additional  missiles,  or  he  could  set 
this  risk  money  aside  to  be  used  as  necessary  in  making  la- 
ter Improvements.  The  method  of  assigning  a dollar  cost 
to  each  risk  allows  the  quantification  of  costs  for  each 
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I 

I 

I 
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o tie:,  o.i  ••  n o ju?il  ri  r.'.c  basis. 

. . her  some  obviou  lvant  f k 

:.a‘  should  be  stressed: 

Risl  analy  is  ri vi  ti<  ntl 

•i*  a:,  value::,  variances,  and  ri  k of  • vc  i in  - 
• ' - ' h . thr<  : s . • ' ■ , ' •• 

can  be  mad e of  what  the  ponsi  ale  .are  d is,  in 
values  or  risks. 

. 1 naly  si  - nsi- 

tivity  analyses  on  the  various  or  lions  to  arrive 
at  the  c p ti..r;:a  solution. 

. ' lalysi  ‘orces  /aluati 

;uole,  rather  than  consideration  of  only 
r when  :ln  ' 

Ris  si  la;  contr  icl  r intuitiv  . 

■ ' ' . ' L,  01 

problem  to  be  reevaluated  and  tor 

tision  to  be  r . ' 

will  not  always  provide  a dominant  ;tr  te.  y,  but 
It  will  provide  the  decision  maker  with  the  pre- 
dicted consequence  of  each  co.rse  of  action. 

If.  One  criticism  of  th<  i roach  to  risl 

in  this  case  study  is  the  use  of  sunk  costs  as  part  of  the 

overall  program  cost.  DCD  INST  TCAl.  3 points  out  that  sunk 
costs  should  have  no  bearing  on  current  management  decisions. 
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Not!  thal  In  ' - . 1 tlons  jel<  , th<  . c >t  Lb  a 
rt  ol  each  option  and  In  the  final  Jud  -ment  of  the  de- 
cision maker  it  does  not  affect  the  relative  outcome  of 
th<  options.  The  risk  for  pro  ;ra  - st  is  assessed  by 

wh<  ther  11  ha?  n<  I the  r<  ran  c<  . t -oai  •.a.ic:.  c b- 

lished  at  the  beginning  of  the  development  phase.  However, 
the  entire  analysis  could  probably  be  constricted  without 
the  use  of  sunk  costs. 
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1.  Cuff,  J.  D.  , "Risk  Analysis",  Crdnance,  Kay-June  1972, 
pp.  496-4Q9.  (UNCLAS) 

lives  an  introduction  to  risk/decision  analysis 
by  presenting  a case  study  to  show  the  structuring  of 
the  risk  matrix  and  the  possible  f igures-of-neri t the 
analyst  can  sse  to  squats  risk  to  a dollar  base* 

2.  Cuff,  J.  D. , Risk-Decision  Analysis  in  A’eapons  System 
Acq  isitions,  »aj  . 11  shed,  16  May  1972*  (”::CLAC) 

T resents  a case  study  in  risk/declslon  analysis 
po  further  demonstrate  the  usefulness  of  a quantifica- 
tion methodology  in  reducing  risk  in  cost,  schedule, 
and  rerfor-.ance  to  a dollar  base. 
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ATTACHMENT  NO.  1 

Risk  Matrix 
for  the 

Caber  Missile  System 


CHARACTERISTIC  CP 

tic::  i 

optici:  11  c 

: ic.  :: 

Time  Risk 

!!ean  time  until  Snber 
jnlts  : tar t replacin' 

Navi  a tor  aits,  nonths 

12 

24 

Probability  of  havinr 

Saber  Am;  .inits  start 
replacing  Ravi  r-~  tor  units 

H . ths  (Pt) , % 

50' 

CQ 

Cost  Risk 

Mean  cost,  through  pro- 
auction  of  5,000  Saber 
missiles  and  100  launch 
vehicles,  in  millions 
r>  f dollars 

510 

560 

570 

Probability  of  not  exceed- 
ing the  cost  -oal  (Fc)  of 
$500  K,  % 

23 

0 

0 

Reliability  Risk 

Mean  reliability  of  field- 
ed missiles,  •' 

70 

75 

75 

Probability  of  raeetinr  re- 
liability yoal  (?)  of  75% 
with  fielded  missiles 

20 

50 

50 

Accuracy  Risk 

Mean  accuracy,  mils  CRF 

1.5 

1.0 

1.0 

Probability  of  neetin,- 
accuracy  goal  (Fa)  of  1. 
mils  CEP,  % 

1 

50 

50 
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ATTACK'S!’? 


::c. 


2 


Decision  Analysis  Results 
for  the 

Saber  !'isslle  System 


r 


' 
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— 

^xrec 
(Costs  in 

_ ... 

ted  Value  Ar 
Millions  of 

rroach 

Dollars) 

CHARACTERISTIC 

optic::  i 

optic::  ii 

cptic::  hi 

Time  cost 

0 

50 

0 

Prorrara  cost 

r\ 

SJ 

50 

60 

Reliability  cost 

39.27 

o 

0 

Accuracy 

3'M.,  .75 

0 

rc  tax  o]  noN  incrs- 

MEMTAL  COST 

405.22 

100 

60 

TOTAL  EXPECTED  OPTIC!. 
cost,  including  #510 
cost,  for  Lowest  Cost 
? ro  < Tara 

915.02 

•:1C  .r 

' . 

i 

Decision  Tree  Approach 
(Equivalent  Tost  in  Millions) 


optic:: 

V T V 

.-.i.  .< 

EXPECTED 

MAX 

i 

600 

915 

1319 

ii 

399 

630 

009 

hi 

425 

607 

869 

I 

I 

I 

I 
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ATTACHMENT  NO.  3 

Sample  Data  Sheets 

for 

Costing  Retrofit  Procedures 
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(Assume  the  nissile  (or  warhead)  is  already  at  the  depot  and  no  missile  (or  war head)  shipping  costs  are  involved.)  At 
a description  of  the  removal  and  replace  actions  necessary  to  acquire  the  item  for  rework.  Hnter  the  values  of  COR  in 
the  table. 
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